Information-gap robustness of a neural network regression model

As a result of their black-box nature, neural networks resist traditional methods of certification and therefore cannot be used in safety critical applications. This situation is undesirable as neural networks can provide an effective solution to many engineering problems. The object of the current paper is to explore the possibility of quantifying and qualifying the reliability of neural networks by a means outside the traditional framework. The approach used here will follow Ben-Haim’s information-gap theory of uncertainty. This is a non-probabilistic approach which may lend itself well to certification of black-box systems. The approach is demonstrated here on a neural network regression model of the process of pre-sliding friction between solids. INTRODUCTION Over the past few years there has been an enormous increase in the number of proposed application areas for Artificial Neural Networks (ANNs) within the sphere of engineering; particularly in control systems, fault diagnostics and “smart structures”. Recent examples include applications to aircraft wing damage detection [1, 2], damage assessment in steel frame structures [3], and the generic use of such systems for changes in structural parameters [4,5]. However, despite intensive activity within both academia and industrial R&D environments, the uptake of ANN technology by industry has been minimal, particularly within Europe. One of the main reasons for this resistance is the apparent “black-box” nature of ANNs which makes them resistant to traditional methods of certification and therefore severely restricts their application to safety-critical systems. For example, within aerospace applications, the decisions made by a neural network would determine if the aircraft should be withdrawn from service for detailed inspection or repair. The consequences of errors by the network are potentially costly and have obvious life-safety implications. There is clearly some need for an evaluation of the network dependability. ANNs often perform well within an environment where the training data sets and test data sets are well matched to each other. However in “real-world” applications it is precisely the unpredictable nature of variations (whether environmental or instrumentation dominated) between what the network has been trained on, and the current data presentation, that causes most concern. A typical practical neural network consists of a non-linear mapping between input and output vectors. Although a huge number of different approaches are possible [6,7], the Multi-Layer Perceptron (MLP) remains probably the most widely used network architecture. An MLP network can model any continuous function, to arbitrary accuracy, provided there is a sufficient number of hidden nodes, and that the network weights and biases are appropriate [8]. It is in the potential complexity of the transfer function, due to the distribution of connection weights combined with the inherent non-linearity of an MLP structure, that the problem of network characterization arises. No matter how extensive the training and testing, there may always be a suspicion that a particular combination of input conditions will arise (as yet not presented to the network) that would lead to unusual and unforeseen outputs. Most statistical methods of reliability assessment rely on the availability of probability distributions and the concept of probability of failure or Mean Time To Failure (MTTF) [9]. However, these probability distributions are usually estimated from the low order moments of the data, typically mean and standard deviation, and do not necessarily represent the tails of the distributions with any accuracy. This represents a problem as the events of interest in reliability are extreme events associated with outliers in the probability distribution. The approach in the current work is to adopt a non-probabilistic approach, based on the theory of convex models and information-gap uncertainty as pioneered by Ben-Haim [10, 11, 12]. The response of a simple MLP network trained on input/output data representing the force/displacement characteristics of a pair of surfaces in the regime of pre-sliding friction is investigated. After conventional training and testing of the network, the propagation of intervalised [13] vectors through the network structure is studied. These interval inputs represent the set of all input conditions that can arise (within certain prescribed limits) for presentation to the network. The intervalised network response therefore provides a conservative estimate of all possible responses to the network and therefore must contain the worst case error scenario. Additionally, if a particular amount of uncertainty can be tolerated, it is possible to reduce the best case error; this represents the opportunity of the system in Ben-Haim’s terminology. NETWORK STRUCTURE The current approach is one founded in system identification, i.e. to identify a model that generates a valid representation between given input and output data. We use an ANN with an MLP structure to provide a nonlinear black-box approach to finding the transfer function between a vector of inputs xi and a vector of outputs yi. The input/output data was modelled by fitting to a NARX model (Non-linear, AutoRegressive with eXogenous inputs), i.e. a particular instantaneous output value ŷi is determined partly by the previous output values yi and partly by previous input values xi [14]. ] ,... ; ,... [ ˆ 1 x y n i i n i i i x x y y F y − − − = Eqn. (1) For a linear system: x x y y n i n i i n i n i i i x b x b x b y a y a y a y − − − − − + + + + + = ... ... ˆ 1 2 1 2 2 1 1 Eqn. (2) The requirement is to find the coefficients a and b that will satisfy the above equations (1) and (2). For a non-linear MLP with a single hidden layer with a non-linear tanh activation function, and an output layer with a linear activation function, the output is given by: